notebook.community

Edit and run

How to read in, visualize and analyze SIF data

Please see http://geo.holoviews.org/user_guide/Gridded_Datasets_I.html for background. A proper python installation is needed and some tools will need to be installed.





In [9]:



    


# https://scitools.org.uk/iris/docs/latest/index.html (data vis tool)
#import iris 
# Make sure plots are create inline
%matplotlib inline
import matplotlib.pyplot as plt
# numpy 
import numpy as np
# xarray (very handy)
import xarray as xr
# specific for the tools here 
# http://geo.holoviews.org/index.html
import holoviews as hv
import geoviews as gv
import geoviews.feature as gf
from cartopy import crs as crrs
import warnings
gv.extension('bokeh')

NetCDF MF datasets can be very powerful as you can use wildcards to access multiple files at once and your code interprets them as "one file", concatenated in the time dimension. NetCDF4 tools can do this and so can xarrays in python with the open_mfdataset command. TROPOMI SIF files are constructed in a way tha these tools can actually be used.





In [3]:



    


# READ in all TROPOMI gridded SIF data 
xr_tropomi = xr.open_mfdataset('/Volumes/data1/ftp/data/tropomi/gridded/TROPO_SIF_0[4567]*.nc')
xr_tropomi



    


















    
Out[3]:








<xarray.Dataset>
Dimensions:     (lat: 900, lon: 1800, time: 122)
Coordinates:
  * lon         (lon) float64 -179.9 -179.7 -179.5 -179.3 -179.1 -178.9 ...
  * lat         (lat) float64 -89.9 -89.7 -89.5 -89.3 -89.1 -88.9 -88.7 ...
  * time        (time) datetime64[ns] 2018-04-01 2018-04-02 2018-04-03 ...
Data variables:
    SIF         (time, lat, lon) float32 dask.array<shape=(122, 900, 1800), chunksize=(30, 900, 1800)>
    dcSIF       (time, lat, lon) float32 dask.array<shape=(122, 900, 1800), chunksize=(30, 900, 1800)>
    relSIF      (time, lat, lon) float32 dask.array<shape=(122, 900, 1800), chunksize=(30, 900, 1800)>
    SIF_err     (time, lat, lon) float32 dask.array<shape=(122, 900, 1800), chunksize=(30, 900, 1800)>
    dcSIF_err   (time, lat, lon) float32 dask.array<shape=(122, 900, 1800), chunksize=(30, 900, 1800)>
    relSIF_err  (time, lat, lon) float32 dask.array<shape=(122, 900, 1800), chunksize=(30, 900, 1800)>
    cont        (time, lat, lon) float32 dask.array<shape=(122, 900, 1800), chunksize=(30, 900, 1800)>
    n_meas      (time, lat, lon) float64 dask.array<shape=(122, 900, 1800), chunksize=(30, 900, 1800)>

















In [63]:



    


# Define dataset "key" dimension:
kdims = ['time', 'lon', 'lat']
# define target "value" dimension:
vdims = hv.Dimension('dcSIF', range=(0, 2))



    











In [52]:



    


hv.Dimension.type_formatters[np.datetime64] = '%Y-%m-%d'
sif = xr_tropomi.dcSIF



    











In [12]:



    


tLat = 42
tLon = -94
iLat = np.argmin(np.abs(35-xr_tropomi.lat))
iLon = np.argmin(np.abs(-90-xr_tropomi.lon))
sif_1d = sif[:,iLat,iLon]
plt.plot_date(xr_tropomi.time, sif_1d,'-')
plt.gcf().autofmt_xdate()



    


















    
























In [68]:



    


# Resample dataset to weekly:
xr_dataset_weekly = xr_tropomi.resample(time="W").mean('time')
xr_dataset_weekly



    


















    
Out[68]:








<xarray.Dataset>
Dimensions:     (lat: 900, lon: 1800, time: 19)
Coordinates:
  * time        (time) datetime64[ns] 2018-04-01 2018-04-08 2018-04-15 ...
  * lon         (lon) float64 -179.9 -179.7 -179.5 -179.3 -179.1 -178.9 ...
  * lat         (lat) float64 -89.9 -89.7 -89.5 -89.3 -89.1 -88.9 -88.7 ...
Data variables:
    SIF         (time, lat, lon) float32 dask.array<shape=(19, 900, 1800), chunksize=(1, 900, 1800)>
    dcSIF       (time, lat, lon) float32 dask.array<shape=(19, 900, 1800), chunksize=(1, 900, 1800)>
    relSIF      (time, lat, lon) float32 dask.array<shape=(19, 900, 1800), chunksize=(1, 900, 1800)>
    SIF_err     (time, lat, lon) float32 dask.array<shape=(19, 900, 1800), chunksize=(1, 900, 1800)>
    dcSIF_err   (time, lat, lon) float32 dask.array<shape=(19, 900, 1800), chunksize=(1, 900, 1800)>
    relSIF_err  (time, lat, lon) float32 dask.array<shape=(19, 900, 1800), chunksize=(1, 900, 1800)>
    cont        (time, lat, lon) float32 dask.array<shape=(19, 900, 1800), chunksize=(1, 900, 1800)>
    n_meas      (time, lat, lon) float64 dask.array<shape=(19, 900, 1800), chunksize=(1, 900, 1800)>

















In [69]:



    


# Create a dataset with geoviews
xr_dataset = gv.Dataset(xr_dataset_weekly, kdims=kdims, vdims=vdims, crs=crrs.Robinson())

#.options(cmap='viridis', colorbar=True, fig_size=200)



    











In [ ]:



    


geo_dims = ['lon', 'lat']
(xr_dataset.to(gv.Image, geo_dims).options(cmap='viridis',colorbar=True,height=300,width=600, projection=crrs.Robinson()) * gf.coastline)



    


















    






/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: divide by zero encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: invalid value encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: divide by zero encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: invalid value encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: divide by zero encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: invalid value encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: divide by zero encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: invalid value encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: divide by zero encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: invalid value encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: divide by zero encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: invalid value encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: divide by zero encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: invalid value encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: divide by zero encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: invalid value encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: divide by zero encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: invalid value encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: divide by zero encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: invalid value encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: divide by zero encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: invalid value encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: divide by zero encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: invalid value encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: divide by zero encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: invalid value encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: divide by zero encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: invalid value encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: divide by zero encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: invalid value encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: divide by zero encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: invalid value encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: divide by zero encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: invalid value encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: divide by zero encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: invalid value encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: divide by zero encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: invalid value encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: divide by zero encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: invalid value encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: divide by zero encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: invalid value encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: divide by zero encountered in true_divide
  x = np.divide(x1, x2, out)
/Users/cfranken/anaconda/envs/py36/lib/python3.6/site-packages/dask/array/numpy_compat.py:28: RuntimeWarning: invalid value encountered in true_divide
  x = np.divide(x1, x2, out)

















In [ ]:

Content source: cfranken/SIF_tools

Similar notebooks:

notebook.community | gallery | about